BioVL2: An Egocentric Biochemical Video-and-Language Dataset
نویسندگان
چکیده
本論文では，生化学分野における一人称の実験映像データセットであるBioVL2データセットを提案する．BioVL2データセットは生化学における4種類の基本的実験に対し，それぞれ8動画撮影した合計32，総時間2.5時間の映像からなるデータセットである．各映像はプロトコルと紐づいており，言語アノテーションとして(1)視覚と言語の対応関係のアノテーション，(2)プロトコル中に現れる物体の矩形アノテーションの2種類のアノテーションを付与している．構築したデータセットの応用例として，本研究では実験映像からプロトコルを自動生成する課題に取り組んだ．定量的，定性的な評価の結果，開発した手法はフレームに映っている物体名をそのままプロトコルとして出力する弱いベースラインと比較して，適切なプロトコルを生成できることを確認した．なお，BioVL2データセットは研究用途に限定してデータセットを公開する予定である．
منابع مشابه
Egocentric Video Biometrics
Egocentric cameras are being worn by an increasing number of users, among them many security forces worldwide. GoPro cameras already penetrated the mass market, and Google Glass may follow soon. As head-worn cameras do not capture the face and body of the wearer, it may seem that the anonymity of the wearer can be preserved even when the video is publicly distributed. We show that motion featur...
متن کاملDetecting Engagement in Egocentric Video
In a wearable camera video, we see what the camera wearer sees. While this makes it easy to know roughly what he chose to look at, it does not immediately reveal when he was engaged with the environment. Specifically, at what moments did his focus linger, as he paused to gather more information about something he saw? Knowing this answer would benefit various applications in video summarization...
متن کاملMSR-VTT: A Large Video Description Dataset for Bridging Video and Language Supplementary Material
When organizing the Microsoft Research Video To Language challenge [1], we found that, in our previously released dataset [10], some sentences annotated by AMT workers are identical in one video clip or very similar in one category. Therefore, to control the quality of data and annotations, as well as the competitions, we removed those simple and duplicated sentences and replaced them with refi...
متن کاملEgocentric Video Personalization in Cultural Experiences Scenarios
In this paper we propose a novel approach for egocentric video personalization in a cultural experience scenario, based on shots automatic labelling according to different semantic dimensions, such as web leveraged knowledge of the surrounded cultural Points Of Interest, information about stops and moves, both relying on geolocalization, and camera’s wearer behaviour. Moreover we present a vide...
متن کاملR-Clustering for Egocentric Video Segmentation
In this paper, we present a new method for egocentric video temporal segmentation based on integrating a statistical mean change detector and agglomerative clustering(AC) within an energy-minimization framework. Given the tendency of most AC methods to oversegment video sequences when clustering their frames, we combine the clustering with a concept drift detection technique (ADWIN) that has ri...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Shizen gengo shori
سال: 2022
ISSN: ['1340-7619', '2185-8314']
DOI: https://doi.org/10.5715/jnlp.29.1106